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Executive  Summary 


Department  of  Defense  depot-level  maintenance  activities  often  experience  delays 
in  obtaining  consumable  parts  for  aging  aircraft  systems  and  components.  As 
part  of  a  concerted  effort  to  reduce  these  delays,  the  Defense  Logistics  Agency 
(DLA)  awarded  a  contract  to  the  Logistics  Management  Institute  (LMI)  for  de¬ 
sign,  development,  and  prototyping  of  Intelligent  Collaborative  Aging  Aircraft 
Parts  Support  (ICAAPS).  LMI,  in  partnership  with  JWK  International,  explored 
the  use  of  data  mining  and  analysis  techniques  to  improve  long-range  projections 
of  depot-level  maintenance  requirements  for  DLA-managed  consumable  parts. 

At  the  Navy’s  request,  the  prototype  ICAAPS  demonstration  focused  on  depot- 
level  maintenance  support  for  C-2  aircraft  at  the  Naval  Air  Depot  (NADEP)  in 
North  Island,  CA.  During  this  demonstration,  operational  and  field-level  mainte¬ 
nance  data  were  correlated  with  NADEP  data  to  provide  a  framework  for  project¬ 
ing  the  DLA-managed  consumable  parts  needed  to  perform  standard  depot-level 
maintenance  (SDLM)  on  C-2  aircraft.  However,  because  the  Navy’s  entire 
C-2  inventory  includes  fewer  than  40  aircraft,  a  paucity  of  relevant  data  curtailed 
the  use  of  automated  data  mining  during  the  ICAAPS  prototype  demonstration. 
Nevertheless,  use  of  ICAAPS  analytical  tools  improved  the  forecast  accuracy  for 
all  consumable  parts  included  in  the  prototype  test  sample.  Consequently,  contin¬ 
ued  use  of  ICAAPS  should  facilitate  more  timely  consumable  parts  support  for 
the  C-2  aircraft  maintained  by  NADEP  North  Island,  enabling  reductions  in  the 
number  and  duration  of  C-2  SDLM  delays  caused  by  consumable  parts  shortages. 

Data  limitations  associated  with  the  C-2  prototype  demonstration  precluded  the 
ICAAPS  project  team  from  ascertaining  if  automated  data  mining  can  be  effec¬ 
tively  used  to  forecast  consumable  parts  requirements.  However,  a  follow-on 
ICAAPS  demonstration  at  NADEP  Cherry  Point,  NC  should  enable  a  more  robust 
evaluation  of  automated  data  mining  capabilities.  Therefore,  decisions  on  Navy¬ 
wide  ICAAPS  implementation  should  be  deferred  until  the  ICAAPS  demonstra¬ 
tion  at  NADEP  Cherry  Point  is  completed. 


1  The  LMI/JWK  ICAAPS  project  team  included  David  A.  Calderwood  (LMI),  Joseph  Callen¬ 
der  II.  (LMI),  Lawrence  P.G.  Forsley  (JWK),  Adrian  Hernandez  (LMI),  Robert  L.  Jordan  (LMI), 
Scott  Phillips  (JWK),  and  Galbraith  D.  Williams  (JWK). 
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Chapter  1 

Introduction 


Overview 

Intelligent  Collaborative  Aging  Aircraft  Parts  Support  (ICAAPS)  is  designed  to 
synergistically  integrate  relevant  data  from  multiple  sources  to  improve  projec¬ 
tions  of  consumable  part  requirements,  as  depicted  in  Figure  1-1.  In  this  context, 
ICAAPS  facilitates  identification  and  analysis  of  operational  and  logistics  support 
data  to  determine  the  factors  that  affect  depot  maintenance  requirements  for  the 
consumable  parts  needed  to  support  aging  aircraft. 

Figure  1-1.  ICAAPS  Data  Integration  Concept 


Objectives  and  Analytical  Methodology 

The  objective  of  this  task  was  to  develop,  demonstrate,  and  document  ICAAPS 
capability  to  accurately  project  consumable  parts  requirements  for  C-2  aircraft 
undergoing  depot-level  maintenance  at  the  Naval  Air  Depot  (NADEP)  in  North 
Island,  CA.  Within  this  context,  the  LMI/JWK  ICAAPS  project  team 

♦  analyzed  the  information  and  procedures  used  to  determine  requirements 
for  consumable  parts  needed  to  support  depot-level  maintenance; 

♦  used  data  mining  and  analysis  to  identify  data  elements  and  relationships 
that  impact  depot-level  maintenance  consumable  part  requirements; 
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♦  designed  ICAAPS  to  complement  and  enhance  current  processes  used  by 
the  Navy  to  project  depot-level  maintenance  consumable  parts  require¬ 
ments; 

♦  performed  a  prototype  demonstration  of  ICAAPS  for  C-2  aircraft 
undergoing  standard  depot-level  maintenance  (SDLM)  at  NADEP  North 
Island; 

♦  identified  appropriate  metrics  for  determining  ICAAPS  effectiveness;  and 

♦  assessed  ICAAPS  accuracy  vis-a-vis  historical  projections  of  consumable 
parts  requirements  in  support  of  C-2  SDLM  at  NADEP  North  Island. 

Report  Structure 

Chapter  2  of  this  report  outlines  the  processes  in  use  by  the  Navy  to  project  future 
depot-level  aviation  maintenance  requirements  for  DLA-managed  consumable 
parts.  Chapter  3  explains  how  ICAAPS  enhances  the  Navy’s  current  consumable 
parts  forecasting  process.  Chapter  4  discusses  the  prototype  ICAAPS  demonstra¬ 
tion,  which  involved  C-2  SDLM  consumable  parts  requirements  and  assesses  the 
demonstration  results.  Chapter  5  summarizes  noteworthy  conclusions  and  rec¬ 
ommendations.  Appendix  A  provides  amplifying  information  about  C-2  data 
structures,  and  Appendix  B  defines  the  abbreviations  used  in  this  report. 
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Chapter  2 

Current  Navy  Forecasting  Process 


Figure  2-1  shows  the  Navy’s  current  process  for  projecting  aviation  depot-level 
maintenance  consumable  parts  requirements.  This  process  involves  compilation 
and  analysis  of  data  generated  by  the  Naval  Air  Systems  Command  (NAVAIR)  at 
Patuxent  River,  MD;  the  Navy  Inventory  Control  Point  (NAVICP)  in  Philadel¬ 
phia,  PA;  and  the  NADEPs  at  Cherry  Point,  NC,  Jacksonville,  FL,  and  North  Is¬ 
land,  CA,  which  generate  special  program  requirements  (SPRs)  for  the  Defense 
Supply  Centers  (DSCs)  at  Richmond,  VA,  Philadelphia,  PA,  and  Columbus,  OH. 

Figure  2-1.  Current  Navy  Process  for  Projecting  Consumable  Parts  Requirements 


Most  of  the  organizations  involved  in  projecting  aviation  depot-level  maintenance 
consumable  part  requirements  have  developed  automated  tools  to  facilitate  the 
complex  probabilistic  computations  that  are  inherent  in  this  process;  however, 
manual  interfaces  are  required  in  many  cases  (i.e.,  Excel  spreadsheets  are  fre¬ 
quently  used  to  convey  requirements  between  different  organizations). 
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NAVAIR 


NAVAIR  forecasts  aircraft  depot-level  maintenance  requirements  annually  to 
achieve  readiness  goals  within  financial  and  schedule  constraints.  These  forecasts 
include  requirements  generated  under  three  different  depot-level  maintenance 
criteria: 

♦  Aircraft  that  undergo  SDLM;  the  forecasts  represent  the  average  number 
of  aircraft  projected  to  fail  Aircraft  Service  Period  Adjustment  (ASPA)  in¬ 
spections 

♦  Aircraft  that  are  maintained  under  the  Navy’s  new  integrated  maintenance 
concept  (IMC);  the  forecasts  include  the  individual  aircraft  bureau  num¬ 
bers  (BUNOs)  that  are  scheduled  for  IMC  events 

♦  Aircraft  involved  in  upgrade  programs  (e.g.,  major  modifications  or  ser¬ 
vice  life  extensions);  the  forecasts  include  the  BUNOs  that  are  scheduled 
for  induction  under  those  programs. 

NAVAIR  uses  the  Depot  Requirements  Determination  Model  (DRDM)  and  the 
Depot  Requirements  Assessment  Model  (DRAM)  to  establish  annual  aircraft 
maintenance  induction  requirements  and  rank  the  priority  of  requirements  based 
on  funding  constraints  and  required  readiness. 


DRDM 


The  DRDM  is  an  Access-based  application  used  to  define  overall  aircraft  mainte¬ 
nance  requirements.  It  identifies  the  annual  quantity  of  aircraft  expected  to  fail 
ASPA  inspections,  undergo  IMC  events,  or  require  depot  maintenance  associ¬ 
ated  with  other  requirements,  such  as  major  modifications.  As  illustrated  in 
Figure  2-2,  DRDM  inputs  include  Aircraft  Inventory  and  Readiness  Reporting 
System  (AIRRS)  data,  primary  aircraft  allowance  (PAA)  information,  IMC 
schedules,  and  probability  tables  on  expected  number  of  ASPA  failures.  DRDM 
outputs  include  ASPA  inspection  schedules,  projected  ASPA  failure  quantities, 
and  scheduled  aircraft  retirements. 
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Current  Navy  Forecasting  Process 


Figure  2-2.  DRDM  Inputs  and  Outputs 
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DRAM 


The  DRAM  is  an  Access-based  application  used  to  determine  the  effects  of  depot 
maintenance  budget  constraints  on  aircraft  readiness.  It  facilitates  optimization  of 
depot-level  maintenance  induction  priorities  by  balancing  available  funding  with 
readiness  requirements  and  aircraft  inventories.  As  Figure  2-3  illustrates,  DRAM 
inputs  include  funded  requirements  and  cost  data  from  the  budget  spreadsheet, 
PAA  levels  from  the  Aircraft  Program  Data  File  (APDF),  and  current  inventory 
levels  from  the  A-II  spreadsheet.  DRAM  outputs  consist  of  a  series  of  reports  that 
assess  readiness  for  a  given  funding  level  and  indicate  the  number  of  depot  induc¬ 
tions  needed  to  meet  readiness  goals. 
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Figure  2-3.  DRAM  Inputs  and  Outputs 
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NAVICP 

NAVICP  forecasts  aviation  depot-level  reparable  (AVDLR)  induction  re¬ 
quirements  based  on  projected  ready-for-issue  (RFI)  inventory  levels.  NAVICP 
negotiates  quarterly  repair  schedules  with  individual  NADEPs  based  on  pro¬ 
jected  availability  of  reparable  carcasses  and  consumable  parts.  NAVICP  also 
transmits  parts  usage  forecasts  and  configuration  changes  to  DLA  so  that  cogni¬ 
zant  item  managers  can  make  appropriate  adjustments  in  future  procurements. 

Key  NAVICP  applications  involved  in  the  AVDLR  consumable  parts  forecasting 
process  include  the  Inventory  Manager  (IM)  Toolkit,  Configuration  Item  Status 
Sheet  (CISS),  and  Configuration  and  Technical  Notification  Program  (CaTNP). 


IM  Toolkit 


NAVICP  uses  the  IM  Toolkit  to  forecast  AVDLR  requirements.  A  2-quarter  fore¬ 
cast  is  used  as  the  basis  for  negotiating  firm  depot  induction  requirements,  subject 
to  the  availability  of  reparable  carcasses.  In  addition,  an  8-quarter  forecast  pro¬ 
vides  long  range  requirements  based  on  historical  carcass  availability  and  known 
future  fleet  requirements.  As  illustrated  in  Figure  2-4,  IM  Toolkit  inputs  include 
unfilled  fleet  requests,  quarterly  forecasted  demands,  and  carcass  availability.  The 
output  consists  of  a  10-quarter  forecast  (2  quarters  of  requirements  used  to  negoti¬ 
ate  near-term  induction  schedules  with  the  NADEPs  and  an  8-quarter,  long-range 
forecast). 
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Current  Navy  Forecasting  Process 


Figure  2-4.  IM  Toolkit  Inputs  and  Outputs 


CISS/CaTNP 

The  CISS  provides  part  configuration  changes  to  DLA  based  on  Design  Change 
Notice  (DCN)  data  regarding  approved  engineering  change  proposals  (ECPs). 
CaTNP  is  a  web-enabled  version  of  the  CISS.  DLA  item  managers  receive  notifi¬ 
cation  of  relevant  changes  by  e-mail,  and  then  log  onto  the  CaTNP  website  to 
view  specific  details.  CaTNP  conveys  the  following  information: 

♦  Part  number  replacements 

♦  Interchangeability 

♦  Disposition  of  material 

♦  SMR  code  changes 

♦  Engineering  remarks 

♦  Cost  avoidance  data 

♦  Weapon  system  designation  codes. 


NADEPS 

The  NADEPs  satisfy  the  SDLM,  IMC,  and  AVDLR  induction  requirements  for 
which  they  serve  as  the  designated  repair  point  (DRP).  SDLM  workloads  and  re¬ 
lated  consumable  parts  requirements  are  subject  to  significant  variation  because 
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they  depend  heavily  on  the  material  condition  of  the  individual  aircraft  that  fail 
ASPA  inspections.  In  contrast,  IMC  consumable  parts  requirements  are  more  pre¬ 
dictable  because  the  IMC  work  content  is  relatively  stable.  It  should  be  noted, 
however,  that  the  consumable  parts  needed  to  complete  “over  and  above”  mainte¬ 
nance  during  IMC  must  be  readily  available  because  the  time  allowed  for  IMC  is 
usually  shorter  than  the  time  allowed  for  SDLM. 

The  NADEPs  use  NAVAIR  and  NAVICP  forecasts  of  aircraft  and  AVDLR  repair 
requirements  to  develop  workload  projections  and  identify  the  consumable  parts 
needed  to  support  those  requirements.  In  recent  years,  the  NADEPs  have  used  the 
following  legacy  applications  to  project  consumable  parts  requirements: 

♦  Cherry  Point  and  Jacksonville — Material  Availability  Forecasting  (MAF) 

♦  North  Island — Depot  Bill  of  Materials  (DepBOM) 

In  addition,  all  NADEPS  are  implementing  a  Manugistics  Com/^mCONTRACT 
manufacturing  resource  planning  (MRP  II)  system  that  includes  maintenance,  re¬ 
pair,  and  overhaul  (MRO)  functionality. 


MAF 


MAF  is  a  flexible  Access-based  application  developed  by  NADEP  Jacksonville 
that  can  be  used  to  compare  projected  consumable  parts  requirements  for  both  air¬ 
craft  and  AVDLRs  with  available  inventories  to  determine  if  additional  procure¬ 
ments  will  be  needed.  MAF  does  not  have  the  functionality  needed  to  create  and 
maintain  Bills  of  Material  (BOMs),  but  it  does  contain  algorithms  that  compare 
projected  consumable  parts  requirements  with  expected  inventory  levels  to  deter¬ 
mine  if  procurement  of  additional  parts  will  be  needed  to  satisfy  projected  re¬ 
quirements,  as  illustrated  in  Figure  2-5. 

Figure  2-5.  MAF  Inputs  and  Outputs 
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Current  Navy  Forecasting  Process 


DepBOM 


DepBOM  is  an  Oracle-based  application  used  by  NADEP  North  Island  to  support 
component  repairs.  As  Figure  2-6  shows,  DepBOM  functionality  can  be  divided 
into  four  distinct  areas:  BOM  data  development  and  maintenance,  repair  and  re¬ 
placement  factor  maintenance,  component  unit  pricing,  and  consumable  part  fore¬ 
casting  and  reporting  (using  MAF  algorithms). 

Figure  2-6.  DepBOM  Inputs  and  Outputs 
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Repair  BOMs  are  developed  and  maintained  within  the  DepBOM  application.  Ac¬ 
tual  parts  usage  information  from  the  CompassC ontract  MRP  II  System  is  used  to 
maintain  consumable  part  repair  and  replacement  factors  within  DepBOM.  Com¬ 
ponent  unit  pricing  information  provides  data  for  competitively  awarded  work¬ 
loads  and  maintains  supporting  data  for  audit  purposes. 


CompassCONTRACT  MRP  II  System 

The  MRP  II  system  contains  dynamic  repair  and  replacement  factors  for  items 
contained  in  the  BOMs  to  accommodate  MRO  operations.  These  repair  and  re¬ 
placement  factors  are  used  in  calculating  time-phased  material  requirements. 
Figure  2-7  shows  the  master  planning  schedule,  inventory  status,  related  BOMs, 
and  other  planning  factors  that  are  CompassCONTRACT  inputs.  The  output  of 
the  MRP  II  system  consists  of  recommended  purchase  orders  and  shop  orders, 
with  exception  messages  where  actions  must  be  taken  to  achieve  the  plan. 
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Figure  2-7 \  CompassCOAT/MCT Inputs  and  Outputs 
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DLA 


DLA  procures  consumable  parts  for  use  by  all  services,  based  primarily  on 

♦  an  order-point  stock  replenishment  methodology  for  items  that  have  well- 
defined  historical  demand  patterns,  and 

♦  numerical  stocking  objectives  (NSOs)  for  items  that,  if  not  available  when 
needed,  would  seriously  impact  military  effectiveness  or  efficiency.  Con¬ 
sequently,  investment  in  such  items  is  warranted  despite  a  high  probability 
that  those  items  will  seldom  be  needed. 

In  addition,  the  services  submit  SPRs  for  items  they  anticipate  will  experience 
significantly  increased  demands.  However,  when  actual  demands  exceed  pro¬ 
jected  requirements  or  deplete  on-hand  inventories,  the  associated  procurement 
delays  often  cause  maintenance  work  stoppages.  As  part  of  a  concerted  effort  to 
alleviate  this  problem,  DLA  tasked  LMI  (in  partnership  with  JWK  International) 
to  design,  develop,  and  prototype  ICAAPS  as  a  tool  for  improving  long-range 
projections  of  depot-level  maintenance  requirements  for  DLA-managed  consum¬ 
able  parts. 
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Chapter  3 

ICAAPS  Forecasting  Process 

ICAAPS  enhances  the  Navy’s  current  process  for  projecting  aviation  depot-level 
maintenance  consumable  parts  requirements.  It  does  this  by  compiling  and  corre¬ 
lating  a  broad  range  of  operational  and  logistics  data  from  the  Navy’s  Aviation 
Maintenance  Material  Management  (AV3M)  system  as  well  as  historical  NADEP 
workload  and  DLA  supply  support  data.  Figure  3-1  illustrates  how  ICAAPS 
data  mining  and  analysis  augments  the  Navy’s  current  process  for  forecasting 
DLA-managed  consumable  parts  requirements. 

Figure  3-1.  ICAAPS  Depot-Level  Consumable  Parts  Forecasting  Process 


Current  projections  of  depot  maintenance  requirements  for  consumable  parts  are 
based  on  historical  depot-level  maintenance  data.  Consequently,  these  projections 
frequently  underestimate  the  range  and  depth  of  consumable  parts  required,  par¬ 
ticularly  for  aging  aircraft  that  experience  unexpected  increases  in  consumable 
parts  requirements.  As  depicted  in  Figure  3-2,  ICAAPS  expands  the  current  plan¬ 
ning  horizon  for  depot-level  maintenance  to  include  relevant  information  gathered 
during  the  entire  operating  service  period. 
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Figure  3-2.  ICAAPS  Versus  Traditional  Depot-Level  Maintenance 

Planning  Horizon 


Aircraft  and  AVDLR 

y  t  Depot-level 

operational  usage 

maintenance 

Current 

< - planning - ► 

_  .  .  .  horizon 

^  ICAAPS  planning  horizon  -  ► 


Because  current  projections  of  depot-level  maintenance  consumable  part  require¬ 
ments  are  based  primarily  on  historical  depot-level  maintenance  data,  there  is  a 
delay  of  about  2  years  from  the  time  unexpected  consumable  part  requirements 
are  first  discovered  to  the  time  when  those  parts  are  reflected  in  projections  of  fu¬ 
ture  requirements.  By  expanding  the  depot-level  maintenance  planning  horizon  to 
include  relevant  information  gathered  during  the  entire  operating  service  period, 
ICAAPS  reduces  this  inherent  forecasting  lag,  as  depicted  in  Figure  3-3. 

Figure  3-3.  ICAAPS  Effect  on  Forecasting  Lag 


Data  Mining  and  Analysis 

ICAAPS  data  mining  and  analysis  involves  the  following  steps: 

♦  Identification  and  collection  of  relevant  operational,  maintenance,  and 
supply  data,  based  on  interviews  with  cognizant  DoD  personnel;  the  fol¬ 
lowing  data  elements  are  valuable  inputs  to  ICAAPS  data  mining  and 
analysis: 

>  Aircraft  logbook  data 

■  Squadron  and  reporting  custodian 

■  Number  of  flight  hours  (lifetime  total  and  since  last  depot  visit) 

■  Number  and  length  of  shipboard  deployments 
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■  Number  of  catapults  and  arrested  landings 

■  Time  spent  out  of  reporting  status  (“hangar  queens”) 

>  Field  maintenance  data 

■  Average  number  of  maintenance  labor-hours  by  work  unit  code 

■  Landing  gear  “drop”  checks 

■  Engine  changes  (lifetime  total  and  number  since  last  depot  visit) 

■  Other  noteworthy  squadron  maintenance  actions 

>  Depot  maintenance  data 

■  Data  and  turnaround  time  of  last  depot  visit 

■  Type  and  location  of  “over  and  above”  repair  work 

■  Size,  location,  and  number  of  bushings  and  oversized  fasteners 
used 

■  Repair  and  replacement  parts  usage 

>  DLA  demand  and  requisition  history  data 

♦  Standardization  of  data  from  disparate  data  sources  so  that  values  and 
definitions  of  data  fields  are  consistent;  some  data  fields  must  be 
“scrubbed”  and  normalized  to  eliminate  errors  and  redundancy 

♦  Development  and  testing  of  various  hypotheses  regarding  potential  cause- 
and-effect  relationships  among  the  operational-  and  field-level  mainte¬ 
nance  factors  that  may  affect  future  depot-level  maintenance  requirements; 
following  is  a  list  of  key  hypotheses: 

>  Configuration — Different  aircraft  configurations  are  likely  to  have  dif¬ 
ferent  material  condition  profiles  and  use  different  consumable  parts 

>  Overall  time  in  service — Aircraft  tend  to  require  more  extensive  main¬ 
tenance  as  they  get  older;  in  fact,  NAVAIR  analysis  of  several  aircraft 
types  has  revealed  that  direct  maintenance  labor-hours  per  flight  hour 
increase  by  about  3  percent  per  year  as  aircraft  age1 

>  Time  elapsed  since  last  depot-level  maintenance — Field-level  mainte¬ 
nance  personnel  do  not  normally  have  the  skills  and  equipment  needed 


1  Naval  Air  Systems  Command,  Aging  Aircraft  Trends  and  Cost  Drivers  Associated  With  Na¬ 
val  Aviation  Systems,  Dr.  Laurence  W.  Stoll,  October  2000. 
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to  accomplish  complex  maintenance  tasks,  particularly  those  tasks  that 
involve  correction  of  corrosion-related  problems  associated  with  com¬ 
posite  structures  and  bonding  between  dissimilar  metals;  consequently, 
the  amount  of  time  since  the  last  period  of  depot-level  maintenance 
should  indicate  how  much  depot-level  maintenance  will  be  required  in 
the  future 

>  Employment  and  utilization — Location  and  intensity  of  operations  may 
have  a  significant  effect  on  aircraft  material  condition  (for  example, 
aircraft  that  operate  for  long  periods  at  sea  tend  to  deteriorate  faster 
than  those  that  operate  primarily  in  less  corrosive  environments);  simi¬ 
larly,  aircraft  that  are  heavily  used  and  experience  numerous  catapults 
and  arrested  landings  generally  require  more  depot-level  maintenance 
than  aircraft  that  experience  less  stressful  usage 

>  Primary  custodian/user — Different  aircraft  custodians  often  have  dif¬ 
ferent  procedures  for  maintaining  their  aircraft,  particularly  for  corro¬ 
sion  control  (for  example,  one  squadron  may  wash  its  aircraft  weekly, 
while  another  squadron  washes  only  them  monthly);  consequently, 
aircraft  material  condition  should  reflect  the  maintenance  practices  of 
aircraft  custodians  just  as  the  material  condition  of  automobiles  re¬ 
flects  the  maintenance  practices  of  their  owners 

>  Field-level  maintenance  or  cannibalization  actions — The  type  and 
quantity  of  field-level  maintenance  performed  on  a  particular  aircraft 
provides  excellent  insights  about  the  overall  material  condition  of  that 
aircraft  (for  example,  a  large  amount  of  field-level  corrosion  control 
work  usually  indicates  that  a  commensurate  amount  of  depot-level 
maintenance  will  be  required  to  correct  corrosion  discrepancies  in  ar¬ 
eas  that  were  not  accessible  to  field-level  maintenance  personnel) 

♦  Identification  of  relevant  “clusters”  of  aircraft  and  AVDLRs  that  have 
similar  material  condition  profiles 

♦  Establishment  of  normalized  forecasts  of  the  consumable  parts  required 
during  depot-level  maintenance  for  specific  clusters  of  aircraft  and 
AVDLRs  based  on  historical  part  usage  data  (for  example,  aircraft  that 
operate  primarily  in  a  shipboard  environment  may  form  a  cluster  that 
has  different  characteristics  than  aircraft  that  operate  primarily  in  a 
shore-based  environment) 

♦  Tailoring  of  projected  consumable  part  requirements  for  specific  aircraft 
and  AVDLRs  based  on  correlations  between  field-level  maintenance  data 
and  specific  depot-level  consumable  part  requirements  (for  example,  if  the 
field-level  maintenance  data  indicates  an  aircraft  windscreen  is  cracked  or 
crazed,  that  windscreen  will  almost  always  be  replaced  during  the  next  pe¬ 
riod  of  depot-level  maintenance). 
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Hardware  and  Software  Tools 

The  initial  ICAAPS  demonstration  performed  by  LMI  and  JWK  was  facilitated  by 
use  of  a  massively  parallel  data  processor  known  as  Silverado,  which  originally 
was  developed  by  JWK  to  process  signal  intelligence  data.  Silverado  is  a  high¬ 
speed  Standard  Query  Language  (SQL)  linearly  scaleable  search  engine  with  a 
scanning  rate  of  up  to  72  million  records  per  second  for  the  current  12  processor 
configuration  (each  processor  can  scan  6  million  records  per  second).  Another 
JWK  data  mining  and  analysis  tool,  HeatSeeker,  rapidly  analyzes  data  relation¬ 
ships  by  automatically  generating  thousands  of  correlation  queries  that  would  take 
much  more  time  using  conventional  database  systems.  In  addition,  BRIO  Intelli¬ 
gence  software  was  used  as  the  front-end  communications  processor  and  desktop 
display  for  the  ICAAPS  prototype.  This  software  enables  data  analysts  to  build 
dynamic  analytical  applications  and  supports  both  pre-defined  and  ad  hoc  queries. 

These  tools  assist  ICAAPS  analysts  in  identifying  relevant  data  relationships  by 
automatically  generating  correlation  queries  that  would  take  much  longer  using 
traditional  data  analysis  procedures.  After  being  identified  by  subject  matter  ex¬ 
perts,  these  relationships  can  be  used  by  depot  and  DLA  personnel  to  develop 
more  accurate  forecasts  of  depot  workloads  and  associated  requirements  for  con¬ 
sumable  parts,  as  depicted  in  Figure  3-4. 

Figure  3-4.  ICAAPS  Reduction  in  Consumable  Part  Forecasting  Time  Lag 
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Chapter  4 

ICAAPS  Prototype  Demonstration 


The  overall  objective  of  the  ICAAPS  prototype  was  to  demonstrate  how  ICAAPS 
can  improve  projections  of  consumable  repair  part  requirements  for  C-2  aircraft 
scheduled  to  undergo  SDLM  at  NADEP  North  Island.  To  accomplish  this  objec¬ 
tive,  we  focused  on  the  following  activities: 

♦  Identification  of  current  practices  and  key  data  sources 

♦  Collection  of  relevant  data 

♦  Evaluation  of  data  structure  and  elements 

♦  Import  of  heterogeneous  data  into  a  common  data  repository 

♦  Use  of  data  mining  and  analysis  to  identify  relationships  between 

>  SDLM  discrepancies  and  consumable  repair  part  usage, 

>  field-level  maintenance  actions  and  SDLM  discrepancies,  and 

>  field-level  maintenance  actions  and  depot-level  part  usage. 

We  conducted  interviews  with  representatives  from  DLA,  NAVAIR  headquarters, 
and  NADEP  North  Island  to  identify  specific  issues  regarding  C-2  SDLM  re¬ 
quirements  for  DLA-managed  consumable  parts.  Our  discussions  with  DLA, 
NAVAIR,  and  NADEP  North  Island  provided  the  following  key  insights: 

♦  NADEP  North  Island  historically  has  been  unable  to  accurately  predict 
consumable  parts  requirements  for  individual  C-2  aircraft  until  the  aircraft 
are  inspected  by  depot  artisans  in  conjunction  with  SDLM  induction. 

♦  Delays  in  obtaining  consumable  parts  often  cause  depot  personnel  to 
“back-rob”  parts  from  other  C-2  aircraft  or  stop  work  while  awaiting  parts. 

♦  Consumable  parts  delays  may  become  more  critical  as  C-2  aircraft  shift 
from  SDLM  to  IMC  because  IMC  events  are  more  time  constrained  than 
SDLM  and  unexpected  “over  and  above”  requirements  have  a  greater  im¬ 
pact  on  cost  and  schedule. 

♦  Because  the  C-2  has  a  shrinking  supplier  base  and  very  low  demand  rates 
for  most  consumable  parts,  the  fill  rate  for  C-2  parts  is  unlikely  to  improve 
unless  better  methods  are  used  to  project  consumable  part  requirements. 
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At  NADEP  North  Island,  aircraft  workload  requirements  are  defined  in  Open 
Plan  project  management  software  and  the  associated  consumable  parts  require¬ 
ments  are  defined  in  Com/?ajsCONTRACT,  but  AVDLR  repair  part  requirements 
continue  to  be  defined  in  DepBOM. 

NADEP  North  Island  personnel  enter  the  negotiated  schedule  for  2  quarters  of 
AVDLR  repairs  into  DepBOM  using  Excel  spreadsheets.  Based  on  this  schedule, 
DepBOM  produces  consumable  parts  requirements  for  2  quarters;  these  re¬ 
quirements  are  forwarded  to  FISC  San  Diego.  Because  NAVICP  also  provides  an 
8-quarter  AVDLR  repair  projection,  DepBOM  develops  an  8-quarter  consumable 
parts  requirements  projection  using  MAF  algorithms.  These  algorithms  assess 
Fleet  and  Industrial  Supply  Center  (FISC)  inventory  data,  DLA  inventory  data, 
and  historical  demand  data  to  identify  parts  where  the  expected  availability 
will  not  satisfy  the  projected  requirement.  When  a  projected  shortfall  is  identi¬ 
fied,  the  additional  requirements  are  forwarded  to  DLA  through  NAVICP  in  the 
form  of  SPRs  that  alert  DLA  to  an  increase  in  the  requirement  for  a  particular 
part.  DLA  then  can  adjust  procurement  quantities  to  meet  projected  demands. 

Key  Data  Sources 

As  previously  noted,  we  hypothesized  that  depot-level  consumable  part  require¬ 
ment  projections  could  be  improved  by  correlating  aircraft  characteristics  and 
field-level  maintenance  events  with  depot-level  discrepancies  and  associated  con¬ 
sumable  parts  usage.  To  analyze  these  relationships,  we  compiled  the  following 
data  from  DLA,  NAVAIR,  and  NADEP  North  Island: 

♦  DLA 

>  Requisition  history 

>  Item  headers 

>  Procurement  history 

>  Contracting  technical  data 

>  Active  contract  data 

>  Weapon  system  data 

♦  NADEP  North  Island 

>  Maintenance  planning  and  execution  data 

>  DepBOM 

>  ASPA  and  SDLM  history 
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♦  NAVAIR 

>  AY3M  data 

>  ECP  data. 

Initial  Data  Evaluation 

It  quickly  became  apparent  that  the  AV3M  and  C-2  SDLM  data  were  particularly 
valuable  in  identifying  cause-and-effect  relationships  between  field-level  mainte¬ 
nance  activities  and  depot-level  consumable  parts  requirements.  (Noteworthy 
AV3M  and  C-2  SDLM  data  structure  issues  are  discussed  in  Appendix  A.) 

The  C-2  SDLM  file  provides  comprehensive  data  regarding  the  material  condition 
of  each  C-2  that  underwent  SDLM  between  September  1988  and  June  2001.  Key 
data  elements  include  all  discrepancies  noted,  aircraft  zones  involved,  corrective 
actions  taken,  and  consumable  parts  used. 

The  AV3M  database  historically  has  had  numerous  data  accuracy  problems  be¬ 
cause  of  the  numerous  organizations  that  provide  AV3M  data;  however,  this  did 
not  cause  serious  problems  during  the  ICAAPS  prototype  demonstration  because 
the  AV3M  data  used  for  ICAAPS  analysis  had  previously  been  “cleansed”  by  the 
Center  for  Naval  Analysis  (CNA).  In  fact,  the  major  problems  we  encountered  in 
that  regard  were  data  gaps  that  resulted  from  the  CNA  cleansing  process.  The  fol¬ 
lowing  overview  provides  insight  into  the  depth  and  breadth  of  AV3M  informa¬ 
tion  available: 

♦  MAF1 — Contains  summary-level  data  on  individual  maintenance  actions 
associated  with  specific  aircraft  BUNOs;  it  also  includes  attributes  such  as 
job  control  numbers  (JCNs),  work  center,  BUNO,  and  job  timestamps 

♦  MAF2 — Contains  more  detailed  data  associated  with  the  maintenance  ac¬ 
tions  identified  in  MAF1;  it  includes  transaction  codes,  work  unit  codes 
(WUCs),  malfunction  codes,  actions  taken,  labor  hours  expended,  and  so 
forth 

♦  RT60 — Contains  information  about  the  repair  and  replacement  parts  asso¬ 
ciated  with  particular  maintenance  actions  identified  in  MAF1 

♦  Flights — Contains  operational  data  such  as  number  of  landings,  flight 
hours,  catapult  shots,  and  arrested  landings  for  each  aircraft  BUNO. 

Extract,  Transform,  and  Load  Process 

One  of  the  most  challenging  aspects  of  data  mining  is  compiling  disparate  data 
from  multiple  sources  in  a  manner  that  can  be  queried  easily.  The  extract, 
transform,  and  load  (ETL)  process  involves  extracting  data  from  its  source, 
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standardizing  and  cleaning  that  data,  and  uploading  the  data  into  target  database 
tables.  Several  iterations  of  data  loading  were  required  during  the  ICAAPS  proto¬ 
type  because  of  situations  in  which  the  rules  for  a  given  data  type  were  violated 
by  the  data  values  (e.g.,  alphanumeric  values  were  rejected  when  inserted  into  a 
field  containing  only  numbers).  Standardization  and  cleaning  of  data  generally 
were  accomplished  following  the  data  load  rather  than  as  part  of  the  ETL  process. 

Automated  Data  Mining  and  Analysis 

Application  of  automated  data  mining  techniques  was  a  fundamental  objective  of 
the  ICAAPS  prototype.  In  this  regard,  we  attempted  to  identify  average  aircraft 
material  condition  profiles  through  analysis  of  relevant  field-level  indicators  to 
predict  depot-level  discrepancies  and  associated  consumable  parts  requirements. 
Our  initial  attempts  to  establish  baseline  material  condition  profiles  through 
automated  data  mining  techniques  were  unfruitful,  however,  because  of  the  small 
population  of  C-2  aircraft  and  limited  amount  of  relevant  data. 

Because  data  mining  applications  work  best  in  a  data-rich  environment,  the  pau¬ 
city  of  C-2  data  represented  a  significant  obstacle  to  the  use  of  automated  data 
mining  techniques.  Consequently,  we  decided  to  adopt  a  more  traditional  analyti¬ 
cal  approach  that  involved  developing  and  testing  various  hypotheses  regarding 
potential  cause-and-effect  relationships. 

C-2  Data  Analysis 

Our  first  objective  was  to  identify  data  correlations  that  provided  a  context  for 
prediction  of  depot-level  consumable  parts  usage  based  on  aircraft  characteristics 
and  field-level  maintenance.  Our  overall  hypothesis  was  that  depot-level  mainte¬ 
nance  discrepancies  can  be  correlated  with  both  consumable  parts  usage  and  field- 
level  maintenance  actions,  as  illustrated  in  Figure  4-1. 

Figure  4-1.  ICAAPS  Data  Analysis  Framework 
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We  first  attempted  to  determine  if  there  was  a  correlation  between  depot-level 
discrepancies  and  associated  consumable  parts  usage.  We  then  attempted  to 
associate  maintenance  activities  that  occurred  in  the  field  with  depot-identified 
discrepancies.  Because  field-level  maintenance  is  categorized  by  WUCs  rather 
than  physical  locations  or  aircraft  zones,  we  needed  a  taxonomy  for  relating 
WUCs  and  aircraft  zones  to  provide  a  linkage  between  field-level  maintenance 
actions  and  depot-level  discrepancies. 

Relating  WUCs  to  Aircraft  Zones 

Our  initial  analysis  indicated  that  a  given  C-2  WUC  could  relate  to  one  or  more 
aircraft  zones.  Some  WUCs  can  be  directly  related  to  a  single  zone  because  they 
have  a  one-to-one  relationship  (e.g.,  WUC  1 1C20  refers  only  to  the  outer  wing 
panel  of  the  aircraft);  however,  we  found  other  WUCs  that  relate  to  more  than  one 
aircraft  zone  (e.g.,  WUC  1 1C6B00  covers  the  entire  cargo  bay  floor,  which  en¬ 
compasses  several  different  zones).  In  these  latter  cases,  we  first  related  a  WUC 
to  specific  consumable  parts,  then  related  those  parts  to  the  aircraft  zone  where 
they  are  used.  By  using  this  approach,  we  were  able  to  associate  specific  field- 
level  actions  with  depot-level  discrepancies  so  we  could  identify  relevant 
cause-and-effect  relationships. 

Correlating  Depot  Discrepancies  and  Consumable  Parts  Usage 

Analysis  of  C-2  SDLM  data  revealed  that  the  number  of  depot-level  discrepancies 
is  highly  correlated  (R2  =  80%)  with  the  number  of  consumable  parts  required  to 
correct  those  discrepancies,  as  depicted  in  Figure  4-2. 

Figure  4-2.  C-2  Depot-Level  Discrepancies  Versus  Consumable  Parts  Usage 

Regression  plot 


Parts  =  -6.96619  +  0.289760  Discrepancies 
S  =  45.9838  R-Sq  =  80.4  %  R-Sq  (adjusted)  =  79.5  % 
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While  this  correlation  suggests  the  number  of  depot  discrepancies  is  an  accurate 
indicator  of  total  consumable  parts  requirements,  it  does  not  provide  any  insight 
about  the  range  and  depth  of  consumable  part  inventories  needed  to  fulfill  those 
requirements.  For  the  C-2  aircraft  that  underwent  SDLM  during  the  1990s,  about 
60  percent  of  the  consumable  parts  were  used  only  once,  and  just  15  percent 
were  used  five  or  more  times,  as  shown  in  Figure  4-3.  Consequently,  forecasting 
the  specific  consumable  parts  required  for  C-2  SDLM  can  be  extremely  challeng¬ 
ing  because  of  the  relatively  small  number  of  parts  with  sufficient  data  to  deter¬ 
mine  cause-and-effect  relationships. 

Figure  4-3.  C-2  SDLM  Consumable  Parts  Usage 

3000 


Usage  frequency 

Identifying  Specific  Drivers  for  Consumable  Parts  Usage 

Even  though  we  identified  a  strong  correlation  between  the  number  of  C-2  SDLM 
discrepancies  and  the  number  of  consumable  parts  used  to  correct  those  discrep¬ 
ancies,  we  were  unable  to  statistically  validate  some  of  our  hypotheses  regarding 
cause-and-effect  relationships  among  the  various  factors  that  may  affect  depot- 
level  maintenance  requirements.  Following  are  some  examples: 

♦  Configuration  was  not  a  relevant  factor  because  all  C-2  aircraft  have  the 
same  configuration. 

♦  Overall  time  in  service  was  not  a  relevant  factor  because  all  current 
C-2  aircraft  were  built  within  a  4-year  period  during  the  late  1980s.  How¬ 
ever,  analysis  of  C-2  SDLM  data  revealed  that  early  SDLM  aircraft  used 
different  parts  than  more  recent  SDLM  aircraft,  as  shown  in  Figure  4-4. 
Consequently,  we  decided  to  focus  our  analysis  on  C-2  aircraft  that  un¬ 
derwent  SDLM  during  the  late  1990s  in  order  to  establish  the  baseline  for 
forecasting  future  part  requirements. 
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Figure  4-4.  C-2  Consumable  Parts  Usage  Over  Time 
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♦  Time  elapsed  since  last  depot-level  maintenance — W e  found  no  statistical 
correlation  (R2<10%)  between  the  amount  of  time  elapsed  since  the  last 
depot-level  maintenance  period  and  C-2  SDLM  consumable  parts  usage. 
However,  because  about  60  percent  of  the  consumable  parts  required  for 
C-2  SDLM  during  the  1990s  were  used  only  once,  we  concluded  there 
were  insufficient  data  to  determine  if  a  cause-and-effect  relationship 
exists. 

♦  Employment  or  utilization — We  also  found  no  statistical  correlation 
(R2<10%)  between  C-2  aircraft  employment  and  utilization  and  SDLM 
consumable  parts  usage,  but  we  concluded  there  were  insufficient  data  to 
determine  if  a  cause-and-effect  relationship  exists. 

♦  Primary  custodian  or  user — There  are  three  different  reporting  custodians 
for  C-2  aircraft:  VRC-30  on  the  West  Coast;  VRC-40  on  the  East  Coast; 
and  VAW-120,  the  C-2/E-2  fleet  training  squadron.  As  illustrated  in 
Figure  4-5,  we  found  that  almost  60  percent  of  the  consumable  parts  re¬ 
quired  to  complete  C-2  SDLM  during  the  late  1990s  were  custodian- 
unique.  We  also  found  some  evidence  of  consumable  part  clustering 
among  the  aircraft  owned  by  each  custodian,  but  there  was  insufficient 
data  to  assess  the  statistical  significance  of  these  clusters. 

Figure  4-5.  C-2  Consumable  Parts  Distribution  by  Aircraft  Custodians 


VAW-120 

aircraft 
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♦  Field-level  maintenance  and  cannibalization  actions — Analysis  of  several 
C-2  SDLM  consumable  parts  revealed  that  useful  cause-and-effect  rela¬ 
tionships  can  be  established  between  field-level  maintenance  actions 
and  depot-level  consumable  parts  usage  in  some  cases.  For  example, 
Figure  4-6  reveals  a  strong  relationship  between  the  number  of  field  main¬ 
tenance  hours  expended  on  cargo  door  hinge  fittings  and  the  likelihood 
that  those  fittings  would  be  replaced  during  SDLM.  Specifically,  most 
C-2  aircraft  that  had  more  than  100  hours  of  field-level  maintenance 
near  the  cargo  door  hinge  fittings  needed  new  fittings  during  SDLM. 
Conversely,  aircraft  that  had  less  than  100  hours  of  field-level  mainte¬ 
nance  near  the  cargo  door  hinge  fittings  did  not  need  new  fittings  during 
SDLM.  While  there  could  be  several  explanations  for  this  phenomenon, 
(e.g.,  excessive  corrosion  or  damage  incurred  during  field-level  mainte¬ 
nance),  the  number  of  field-level  maintenance  hours  is  a  reliable  indicator 
of  whether  new  fittings  are  likely  to  be  needed  during  SDLM.  Because  our 
analysis  revealed  similar  relationships  for  several  other  consumable  parts, 
we  concluded  that  field-level  maintenance  actions  can  be  used  to  predict 
depot-level  consumable  parts  requirements  in  selected  cases. 

Figure  4-6.  Field-Level  Maintenance  on  C-2  Cargo  Door  Hinge  Fittings 
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Y  =  fitting  was  replaced  during  SDLM 
N  =  fitting  was  not  replaced  during  SDLM 
—  Empirical  replacement  threshold 


Using  ICAAPS  to  Forecast  C-2  Consumable  Part  Requirements 

Because  we  could  identify  a  relationship  between  C-2  aircraft  belonging  to  differ¬ 
ent  custodians  and  the  consumable  parts  used  on  those  aircraft  during  SDLM,  we 
concluded  that  ICAAPS  can  be  used  to  tailor  depot-level  maintenance  BOMs  to 
reflect  parts  usage  profiles  for  each  aircraft  custodian.  For  example,  Table  4-1 
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ICAAPS  Prototype  Demonstration 

shows  the  average  part  usage  rates  associated  with  aircraft  from  VRC-30, 

VRC-40  and  VAW-120  for  a  representative  sample  of  C-2  SDLM  consumable 
parts. 


Table  4-1.  C-2  Part  Usage  Rates  for  Different  Aircraft  Custodians 


VAW-120 

VRC-30 

VRC-40 

Part  number 

Nomenclature 

Amt. 

Rate 

Amt. 

Rate 

Amt. 

Rate 

123AM40034-15 

Ramp  level  cam  indicator 

0 

0.00 

0 

0.00 

5 

0.56 

123BM40338-601 

Cargo  door  hinge  fitting 

0 

0.00 

3 

0.38 

3 

0.33 

123B1 0466-501 

Pilot  front  windshield 

1 

0.33 

1 

0.13 

2 

0.22 

123H 10074-501 

R/H  wing  latch  cylinder 

1 

0.33 

0.25 

2 

0.22 

123PM1 1407-601 

Throttle  control  bracket 

0 

0.00 

0.38 

1 

0.11 

123WM1 0295-601 

MLG  door  hinge  support 
fitting 

0 

0.00 

1 

0.13 

1 

0.11 

123WM1 0523-601 

Inbd.  engine  mount  support 
fitting 

2 

0.67 

1 

0.13 

6 

0.67 

Note:  Inbd.  =  inboard;  MLG  =  main  landing  gear;  R/H  =  right  hand. 


We  also  found  that  forecasts  for  some  C-2  SDLM  consumable  parts  can  be  re¬ 
fined  by  identifying  cause-and-effect  relationships  with  specific  field-level  main¬ 
tenance  actions.  For  example,  Table  4-2  shows  how  the  expected  requirements  for 
a  representative  sample  of  consumable  parts  can  be  tailored  to  reflect  the  material 
condition  of  C-2  BUNO  162143  owned  by  VRC-40. 

Table  4-2.  Tailoring  C-2  Part  Usage  Rates  for  Individual  Aircraft 


Part  number 

Nomenclature 

VRC-40 
average  usage 
rate 

BUNO  162143 
expected  usage 

123AM40034-15 

Ramp  level  cam  indicator 

0.56 

Yes 

123BM40338-601 

Cargo  door  hinge  fitting 

0.33 

Yes 

123B1 0466-501 

Pilot  front  windshield 

0.22 

No 

123H 10074-501 

R/H  wing  latch  cylinder 

0.22 

Yes 

123PM1 1407-601 

Throttle  control  bracket 

0.11 

No 

123W  Ml  0295-601 

MLG  door  hinge  support  fitting 

0.11 

No 

123WM1 0523-601 

Inbd.  engine  mount  support  fitting 

0.67 

Yes 

The  consumable  parts  usage  projections  for  BUNO  162143  reflected  in  Table  4-2 
are  based  on  the  following  information: 

♦  Ramp  level  cam  indicator — This  part  normally  is  replaced  only  on  aircraft 
from  VRC-40.  Because  BUNO  162143  is  a  VRC-40  aircraft,  and 
AV3M  records  indicate  that  field-level  maintenance  has  been  performed 
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in  the  area  near  the  ramp  level  cam  indicator,  the  indicator  probably  will 
need  to  be  replaced  during  the  next  SDLM,  based  on  ICAAPS  analysis 
that  revealed  evidence  of  a  cause-and-effect  relationship  between  field- 
level  maintenance  and  depot-level  indicator  replacement. 

♦  Cargo  door  hinge  fitting — Because  BUNO  162143  had  almost  150  hours 
of  field-level  maintenance  in  the  area  near  the  cargo  door  hinge  fittings, 
those  fittings  probably  will  be  replaced  during  the  next  SDLM  (100  hours 
of  field-level  maintenance  is  the  empirical  threshold  for  replacing  cargo 
door  hinge  fittings  during  SDLM). 

♦  Right  hand  wing  latch  cylinder — BUNO  162143  data  analysis  revealed 
that  field-level  work  repeatedly  was  performed  in  the  wing  fold  area, 
which  normally  indicates  that  the  wing  latch  cylinder  will  need  to  be  re¬ 
placed  during  SDLM. 

♦  Throttle  control  bracket — BUNO  162143  data  analysis  revealed  that  no 
adjustments  were  made  to  the  throttle  cables  by  field-level  maintenance 
personnel,  which  normally  indicates  that  the  bracket  is  within  limits  and 
will  not  need  to  be  replaced  during  SDLM. 

♦  Main  landing  gear  door  hinge  support  fitting — Although  the  main  landing 
gear  doors  on  BUNO  162143  were  replaced  by  field-level  maintenance 
personnel,  the  malfunction  code  profile  did  not  indicate  any  problems  with 
the  door  hinges.  Consequently,  the  hinge  support  fittings  probably  will  not 
need  to  be  replaced  during  SDLM. 

♦  Inboard  engine  mount  support  fitting — Because  BUNO  162143  had  more 
than  250  hours  of  field-level  maintenance  in  the  area  near  the  inboard  en¬ 
gine  mount  support  fittings,  at  least  one  of  those  fittings  probably  will  be 
replaced  during  the  next  SDLM  (100  hours  of  field-level  maintenance  is 
the  empirical  threshhold  for  replacement  of  engine  mount  support  fittings 
during  SDLM). 

C-2  Prototype  Assessment 

To  assess  ICAAPS  accuracy,  we  compared  ICAAPS  forecasts  for  a  representative 
sample  of  C-2  consumable  parts  against  (1)  a  baseline  projection  reflecting  his¬ 
torical  C-2  SDLM  usage  rates  and  (2)  the  actual  usage  rates  for  the  sample  parts 
during  C-2  SDLMs  that  were  performed  in  the  1999-2001  timeframe.  As  shown 
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ICAAPS  Prototype  Demonstration 

in  Figure  4-7,  the  ICAAPS  forecast  was  more  accurate  than  the  baseline  projec¬ 
tion  for  each  of  the  consumable  parts  depicted  in  Tables  4-1  and  4-2. 

Figure  4-7.  Sample  Part  Forecast  Comparison 
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Chapter  5 

Conclusions  and  Recommendations 


The  ICAAPS  prototype  demonstration  for  C-2  aircraft  undergoing  SDLM  at 
NADEP  North  Island  initially  focused  on  automated  mining  and  analysis  of  nu¬ 
merous  data  elements  to  discover  cause-and-effect  relationships  that  could  be 
used  to  predict  depot-level  discrepancies  and  associated  consumable  parts  re¬ 
quirements.  However,  because  the  paucity  of  C-2  data  severely  limited  the  use  of 
automated  data  mining  techniques,  we  adopted  a  more  traditional  analytical  ap¬ 
proach  that  involved  developing  and  testing  of  hypotheses  regarding  suspected 
cause-and-effect  relationships  between  various  operational  factors,  field-level 
maintenance  actions,  and  depot-level  consumable  parts  requirements.  This  ana¬ 
lytical  approach  produced  the  following  results: 

♦  We  identified  a  strong  correlation  between  the  number  of  C-2  SDLM  dis¬ 
crepancies  and  the  number  of  consumable  parts  used  to  correct  those  dis¬ 
crepancies,  but  we  were  unable  to  statistically  validate  several  of  our 
hypotheses  regarding  the  operational  factors  that  may  affect  depot-level 
consumable  part  requirements.  Even  so,  the  ICAAPS  prototype  demon¬ 
strated  how  information  regarding  aircraft  custodians  and  field-level  main¬ 
tenance  actions  can  be  used  to  improve  forecasts  of  the  DLA-managed 
consumable  parts  required  during  C-2  SDLM. 

♦  ICAAPS  improved  the  forecast  accuracy  for  each  of  the  C-2  SDLM  con¬ 
sumable  parts  included  in  the  prototype  test  sample.  Consequently,  ongo¬ 
ing  use  of  ICAAPS  should  facilitate  more  timely  consumable  parts  support 
for  the  C-2  aircraft  maintained  by  NADEP  North  Island,  enabling  reduc¬ 
tions  in  the  number  and  duration  of  C-2  SDLM  delays  caused  by  consum¬ 
able  parts  shortages. 

The  data  limitations  involved  in  this  prototype  demonstration  precluded  an  as¬ 
sessment  of  whether  ICAAPS  can  be  effectively  used  to  forecast  consumable  part 
requirements  Navy-wide.  However,  insights  gained  from  the  C-2  analysis  should 
facilitate  conducting  a  more  robust  follow-on  ICAAPS  demonstration  involving 
H-46  aircraft  and  selected  AVDLRs  (dynamic  components  and  mission  avionics) 
at  NADEP  Cherry  Point. 

In  view  of  the  fact  that  much  more  data  is  available  for  the  H-46  than  for  the  C-2, 
the  ICAAPS  demonstration  at  NADEP  Cherry  Point  should  provide  sufficient  in¬ 
sight  to  assess  the  efficacy  of  using  automated  data  mining  techniques  to  fore¬ 
cast  depot-level  maintenance  consumable  part  requirements.  In  accordance,  we 
recommend  deferring  consideration  of  Navy-wide  ICAAPS  implementation  until 
completion  of  the  ICAAPS  demonstration  at  NADEP  Cherry  Point. 
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Appendix  A 

Import  and  Audit  of  C-2  Data 


Overview 

Data  from  the  Navy’s  Aviation  Maintenance  and  Material  Management  (AV3M) 
system  and  the  Naval  Air  Depot  at  North  Island,  CA,  were  acquired,  cleansed, 
and  imported.  This  data  was  compiled  to  forecast  consumable  parts  usage  dur¬ 
ing  C-2  standard  depot-level  maintenance  (SDLM)  based  on  various  operational 
and  field-level  maintenance  characteristics  of  C-2  aircraft  during  the  past  decade. 
The  overall  data  structure  is  depicted  in  Figure  A-l. 

Figure  A-l.  ICAAPS  Data  Structure 


Data  Definition  and  Audit 

The  first  task  was  to  create  table  definitions  that  reflected  more  accurately  the  ac¬ 
tual  data  provided.  Given  the  paucity  of  detailed  metadata  available,  we  used  an 
iterative  approach  to  import  the  data  into  a  given  table  definition  and  we  used  the 
error  reporting  built  into  the  IMPORT  utility  to  identify  format  problems.  At  the 
end  of  each  iteration,  we  corrected  the  reported  problems  and  then  re-ran  the  data 
until  no  additional  problems  were  reported.  Typical  problems  included 

♦  different  data  types  (field  numeric,  character,  date,  or  timestamp), 

♦  inconsistent  data  formats  (MM/DD/YYYY  or  DD/MM/YY),  and 

♦  invalid  data  fields  (outside  valid  range  for  a  particular  field). 
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Our  approach  was  to  browse  the  source  data  (provided  in  CSV  files,  often  ex¬ 
ported  from  an  Access  database)  and  apply  the  following  rules: 

♦  Fields  within  quotes  (or  double-quotes)  are  character  fields. 

♦  If  several  consecutive  character  fields  are  all  two  characters  (xx)  in  length, 
define  as  char(xx)  and  see  if  the  IMPORT  process  shows  errors  because  of 
longer  fields.  If  a  given  field  repeatedly  comes  up  with  errors,  redefine  as 
something  like  VARCHAR(250)  for  efficiency. 

♦  Numeric  fields  without  delimiting  quotes  are  truly  numeric. 

♦  If  numeric  fields  contain  a  decimal  point,  they  are  defined  as  DECIMAL  to 
allow  the  system  to  retain  the  decimal  places. 

After  we  created  a  CREATE  TABLE  statement  based  on  these  rules,  we  ran  the 
IMPORT  process  and  watched  for  errors.  We  then  corrected  any  errors  and  re¬ 
created  the  table,  continuing  this  process  until  all  errors  either  were  corrected  or 
considered  acceptable. 

When  error-free  tables  were  fully  loaded,  we  ran  the  AUDIT  process  to  provide  an 
easy-to-use  method  of  reviewing  data  content  and  provide  insight  for  further  op¬ 
timization  of  the  table  definition. 

C-2  SDLM  File 

This  was  our  most  complex  file  with  the  largest  number  of  data  fields. 

Many  fields  had  relatively  long  character  data;  thus,  we  defined  them  as 

VARCHAR(xxx). 

This  data  contained  a  time  stamp,  but  the  hours,  minutes,  and  seconds  were  all 
zeros,  so  we  converted  them  to  a  pure  date  format  MM/DD/YYYY. 

There  were  some  DECIMAL  fields,  and  we  defined  them  accordingly;  however, 
some  decimal  places  contained  all  zeroes. 

The  AUDIT  output  shows  that  some  of  the  numeric  INTEGER  fields  could  be  rede¬ 
fined  to  save  space. 


MAF1 


This  file  contained  564,133  data  records,  and  incorporated  a  key  that  was  a  con¬ 
catenation  of  four  job  control  number  (JCN)  fields  (JCNOrg,  JCNDay,  JCNSer, 
and  JCNSuf).  Initial  attempts  to  concatenate  these  fields  showed  that  in  some 
cases  the  JCN  suffix  (JCNSuf)  was  null;  therefore,  any  concatenated  JCN  string 
with  a  null  JCN  suffix  also  was  null.  To  work  around  this,  we  created  files  called 
AV3M_RT60_CONCAT  and  AV3M_MAF1_CONCAT,  containing  both  the  original 
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JCN  fields  and  a  concatenated  JCN  where  any  null  JCNSuf  data  was  changed  to 
xxx  to  ensure  that  the  concatenated  data  retained  the  contents  of  the  first  three 
fields.  This  file  contained  time  stamps  with  both  date  and  time  fields  with  real 
data 


MAF2 

This  was  the  biggest  data  file  with  8,370,563  records.  It  was  a  straightforward 
IMPORT.  The  MAFRECKEY  field  joins  it  to  the  MAF1  file,  and  some  DECIMAL 
fields  have  significant  decimal  data,  but  others  do  not. 

RT60 


The  RT60  file  contains  98,650  records,  and  it  is  related  to  the  MAF1  data  through 
the  combined  JCN  field.  RT60  data  can  be  joined  to  MAF1  data  through  the  com¬ 
bined  JCN.  Initial  attempts  to  concatenate  these  fields  showed  that  in  some  cases 
the  JCN  suffix  (JCNSuf)  was  null  in  RT60,  making  any  concatenated  JCN  string 
with  a  null  JCN  suffix  also  null.  To  work  around  this,  we  created  views  called 
AV3M_RT60_CONCAT  and  AV3M_MAF1_CONCAT  containing  both  the  original 
JCN  fields  and  a  concatenated  JCN  where  any  null  JCNSuf  data  was  changed  to 
xxx  to  ensure  the  concatenated  data  retained  the  contents  of  the  first  three  fields. 


FLIGHTS 


There  were  75,908  flight  records,  each  with  a  unique  FLTRECKEY.  The  date  fields 
were  all  “true  time  stamps”  containing  hours,  minutes,  and  seconds. 

Data  Analysis 

Following  the  initial  analysis  of  each  table  in  isolation,  we  built  a  script  designed 
to  investigate  the  inter-table  relationships.  The  script  ran  five  pieces  of  Standard 
Query  Language  (SQL)  against  every  pair  of  tables  with  a  logical  “join”  relation¬ 
ship.  If  the  two  tables  involved  are  called  A  and  B,  and  the  join  key  is  called  J,  the 
script  will  perform  the  following  actions: 

♦  Count  records  and  unique  occurrences  of  J  in  Table  A. 

♦  Count  records  and  unique  occurrences  of  J  in  Table  B. 

♦  Count  records  and  unique  occurrences  of  J  in  both  Tables  A  and  B. 

♦  Count  records  and  unique  occurrences  of  J  in  Table  A  but  not  in  Table  B. 

♦  Count  records  and  unique  occurrences  of  J  in  Table  B  but  not  in  Table  A. 
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Figure  A-2  depicts  the  annotated  C-2  data  structure. 


BUNO 
31  common 


Figure  A-2.  Annotated  Data  Structure 
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Appendix  B 

Abbreviations 


AIRRS 

Aircraft  Inventory  Readiness  Reporting  System 

APDF 

aircraft  program  data  file 

ASPA 

Aircraft  Service  Period  Adjustment 

AV3M 

Aviation  Maintenance  and  Material  Management 

AVDLR 

aviation  depot-level  reparable 

AWP 

awaiting  parts 

BOM 

bill  of  material 

BUNO 

bureau  number 

CaTNP 

Configuration  and  Technical  Notification  Program 

CISS 

Configuration  Item  Status  Sheet 

CNA 

Center  for  Naval  Analysis 

DCN 

document  control  number 

DepBOM 

Depot  Bill  of  Material 

DLA 

Defense  Logistics  Agency 

DOD 

Department  of  Defense 

DRAM 

Depot  Requirements  Assessment  Model 

DRDM 

Depot  Requirements  Determination  Model 

DRP 

depot  repair  point 

DSC 

Defense  Supply  Center 

ECP 

engineering  change  proposal 

ETL 

extract,  transform  and  load 

FISC 

Fleet  and  Industrial  Supply  Center 

ICAAPS 

Intelligent  Collaborative  Aging  Aircraft  Parts  Support 

IM 

inventory  manager 

IMC 

Integrated  Maintenance  Concept 

JCN 

job  control  number 

MAF 

Material  Availability  Forecasting 

MAF1 

Maintenance  Action  Form  1 

B-l 


MAF2 

Maintenance  Action  Form  2 

MRO 

maintenance,  repair  and  overhaul 

MRP  n 

manufacturing  resource  planning 

NADEP 

Naval  Air  Depot 

NAVAIR 

Naval  Air  Systems  Command 

NAVICP 

Navy  Inventory  Control  Point 

NSO 

numerical  stocking  objective 

PAA 

primary  aircraft  allowance 

RFI 

ready  for  issue 

SDLM 

standard  depot  level  maintenance 

SMR 

source,  maintainability,  and  repairability 

SPR 

special  program  requirement 

SQL 

Standard  Query  Language 

WUC 

work  unit  code 
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